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DETAILED ACTION 

Acknowledgement of Amendments 

The amendment filed on 03/03/2008 overcomes the following 
rejection(s)/objection(s): 

The objection of claim 10 for having minor informalities has been withdrawn in 
view of applicant's amendment. 

Response to Arguments 

1 . Applicant's arguments with respect to claims 1-6 and 8-13 and have been 
considered but are moot in view of the new ground(s) of rejection. 

2. As to applicants argument in regards to Fair does not use or suggest, pose 
estimation, but uses ranging data and the robots pitch/orientation data to operate. 

3. The examiner respectfully disagrees. 

4. Pose estimation is used to identify a specific objects and to determine the 
position and orientation relative to a coordinate system. Fair discloses to a technique 
for detecting the top and bottom of a staircase to allow automated descents and ascent 
of staircases. Additionally, the detection algorithm allows the robot to easily align itself 
with the stairwell, see introduction. Further, in order to verify that it has indeed faced a 
staircase and not a wall, the robot tilts its laser up and down the staircase, calculating 
the average gradient directly ahead. If this matches an acceptable staircase, with an 
angle between 20 and 65 degrees, the robot accepts the situation and moves to the 
next step, see ascent. Therefore, it is apparent to the examiner that Fair more than fairly 
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suggest the use of pose estimation with a staircase model, as it would be necessary for 
the for the robot to negotiate staircases, see staircase negation and table 1 and fig. 1-3. 

5. As applicant arguments, pg 6 paragraph 3. The reference relied upon more than 
fairly suggest and teach the limitations as illustrated in the claim language. Further the 
reference more than adequately teach a monitoring system for the detection of 
obstacles and persons with the use of staircase pose estimation. 

Claim Rejections - 35 USC §112 

6. The following is a quotation of the second paragraph of 35 U.S. C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

7. Claims 1,10, and 12 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 

8. As to claim 1,10, and 1 2, it is unclear as to what is to be considered a geometric 
invariant. 

9. Claims 2-6, and 8-13 fail to remedy the situation stated above, thus claims 2-6, 8- 
9,1 1 , and 13 are also rejected as being indefinite. 

Claim Rejections - 35 USC § 103 

1 0. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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1 1 . The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

12. Claims 1-2 , 4-13 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ponsot et al., US- 6, 606,538 in view of Konolige et al., US 2005/01 00207A1 and 
further in view of Fair et al., Automated Staircase Detection, Alignment & Traversal. 

Re claim 1, Ponsot teaches a monitoring system (figs. 1-2) for the detection a 
person or object within a detection zone of an escalator (col. 3, lines 42-56) comprising 
at least one video camera (5-9) and at least one escalator and/or moving walk (1 ) for 
acquiring a plurality of images. Since Ponsot discloses the detection of persons or 
objects from in a detection zone, it would be inherent to segment the images into an 
escalator background and an obstacles and persons foreground; which is evident by 
Ponsot disclosing detecting the presence of persons or objects in a detecting zone 
(column 1 line 58-66). The cameras 5 to 9 are connected to a monitoring device 10 to 
which they transmit video images of the respective zones they cover. The monitoring 
device is designed to respond to the images transmitted by the cameras 5 to 9 to 
determine whether an object or person is to be found in the detection zone (column 3 
line 50-56). Ponsot is silent in regards to a model based staircase pose estimator, 
acquires stereoscopic images; and determines differences in a rectified stereo images 
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pair which are segmented into an escalator background and an obstacles and persons 
foreground. However, Ponsot fails to teach acquiring stereoscopic images as claimed 
and determining the differences in rectified stereo image pair. And measuring the pixel 
difference between warped and unwarped images wherein image differences are 
represented in an image pyramid. It is noted however that the system and process for 
acquiring stereoscopic images is notoriously well known and used in the art for depth 
perception to enhance image detection as evidence by Konolige. Konolige discloses 
the two images to be correlated may come either from two different cameras separated 
spatially that capture images at the same time ([0002], [0014], [0015], [0017], [0058], 
[0059], and fig. 1 ). Konolige discloses a disparity map calculator connected to the 
output of the stereo correlation means, for calculating a disparity map of the object 
scene ([001 9]). It should be noted that disparity is the difference of two images that 
have been positioned or overlaid on top of one another in stereoscopic or stereovision. 
Further disclosed are Laplacian image pyramids are built and disparity maps are 
produced ([0018]). Further, disclosed is an output at which digital representations of 
pixels in the first video image appear; a second digitizer for digitizing the second video 
image having an input connected to an output of the second video camera, and having 
an output at which digital representations of pixels in the second video image appear; a 
video processor for successively producing sequential stereo Laplacian pyramid images 
at left and right stereo outputs thereof from the digital representations of the first and 
second video images at first and second inputs connected to the outputs of the first and 
second digitizers [0019]. Therefore, it is clear to the examiner that Konolige discloses to 



Application/Control Number: 10/700,266 Page 6 

Art Unit: 2621 

represent images in an image pyramid from the difference of two images, which reads 
upon the claimed limitation. 

One aspect of the pyramid method is progressive compression, which the 
decoder inputs the compressed stream section by section, and each section improves 
the appearance of the image; which would also rectify the image. The examiner takes 
the position that rectification is just the adjustment of images to simplify stereoscopic 
images. Konolige also discloses the use of rectification for images that due to lighting 
biases that differ between the images, distortions introduced by the lenses, and image 
plane geometry. Compensation for geometric distortion is possible by rectifying the 
original images, by mapping the original image into a warped image ([0078]). Also, 
Konolige discloses the use of sub-pixel mapping between the rectified and original 
image ([0086] and [0087]). The examiner takes the position that in order for the method 
as disclosed by Konolige to compute sub-pixel mapping between the rectified image 
and the original image, it would necessitate the comparing and determining the 
differences of the rectified image. 

However, the combination of Ponsot and Konolige are silent in regards to a 
model based staircase pose estimator based upon grouping of line features by the use 
of geometric invariant; and segmenting into a staircase background and obstacle 
foreground. However, Miller discloses a technique for detecting the top and bottom of a 
staircase to allow automated descents and ascents of staircases (see introduction) as 
well as stair negotiation. Further, in order to verify that it has indeed faced a staircase 
and not a wall, the robot tilts its laser up and down the staircase, calculating the average 
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gradient directly ahead. If this matches an acceptable staircase, with an angle between 
20 and 65 degrees, the robot accepts the situation and moves to the next step, see 
ascent. Therefore, it is apparent to the examiner that Fair more than fairly suggest the 
use of pose estimation with a staircase model to detect background and foreground, 
see fig. 3, as it would be necessary for the for the robot to negotiate staircases, see 
staircase negation and table 1 and fig. 1-3. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine the teachings of Ponsot and Konolige with the 
teachings of Miller for providing position and orientation of objects or persons within the 
detection zone as wells a providing a more intelligent and robust layer of control that 
can plan ahead, bringing safety and efficient to the task of climbing stairs (see 
introduction). 

Re claim 2, the combination of Ponsot, Konolige and Fair as a whole teaches 
everything as claimed above, see claim 1. In addition, Ponsot further teaches the 
video cameras are located above the escalator and/or moving walk (Ponsot, fig. 1). 

Re claim 4, the combination of Ponsot, Konolige, and Fair teaches everything as 
claimed above, see claim 1 . In addition, Ponsot further teaches more than one pair of 
video cameras are arranged along the escalator (moving staircase) and/or moving walk 
to monitor a full length (entire detection zone which includes all of the moving staircase) 
of the escalator and/or moving walk (Ponsot col. 3 line 44-49 also fig. 1 ). 

Re claim 5, the analysis and rejection for claim 1 also apply here. Specially, 
Ponsot fails to further teach a processing unit for processing the stereoscopic images. 
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However, Ponsot does teach the monitoring system comprising a processing unit for 
processing a plurality of images and acquiring images on a moving escalator (column 3 
lines 42-56 and fig. 1). Konolige discloses acquiring stereoscopic images (stereo vision, 
[0004], [0005], 0014], [0015] and fig. 1), and a signal processing unit (fast DSP 
processor, [0033] and 84) for processing stereoscopic images. 

Therefore, the combined teaching of Ponsot and Konolige as a whole would 
have rendered obvious to one of ordinary skill in the art at the time the invention was 
made to acquiring and signal processing stereoscopic from the plurality of images as 
claimed for the benefit of depth perception to enhance image detection as taught by 
Konolige. 

Re claim 6, the combination of Ponsot, Konolige and Fair teach everything as 
claimed above, see claim 5. In addition, Ponsot further teaches the monitoring system 
further comprises a data exchange bus for linking the video cameras with the 
processing unit (column 4 line 17-30, fig. 2: 14, 15) and means for storing the 
stereoscopic images (Ponsot, column 4 line 1-6, 21-24 and fig. 3). 

Re claim 8, the combination of Ponsot, Konolige and Fair as a whole further 
teaches everything as claimed above, see claim 5. Further, Ponsot further teaches the 
processing unit is integrated with at least one camera (Ponsot, col. 8 line 52-56 and fig. 
2). 

Re claim 9, the combination of Ponsot, Konolige and Fair as a whole further 
teaches everything as claimed above, see claim 5. Further, Ponsot further teaches the 
monitoring system is connected electrical to a control for restarting the escalator and/or 
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moving walk after a stop only when no obstacle and/or person is detected on the 
escalator and/or moving walk (Ponsot, col. 2 line 19-23, col. 3 line 56-52, and col. 9 line 
3-6, i.e. switching the unavailability of failure signal of the monitoring device). 

Re claims 10-11, the analysis and rejection made in claims 1-9 also apply here. 
The combination of Ponsot and Konolige as a whole teaches a processor-based 
system. Hence, a computer program product for executing the necessary steps 
corresponding to the system of claim 1 would have been inherent. 

Re claims 12-13, which recite a corresponding method to the monitoring system 
of claims 1-9. Thus, the analysis and rejection made in claims 1-9 also apply here 
because the monitoring system in claims 1-9 would have necessarily performed the 
method steps in claim 12. 

In further regards to claim 13, the combination of Ponsot and Konolige teaches 
restarting the escalator and/or moving walk automatically (Ponsot, col. 4 line 9-16; in 
Ponsot, the main function of the processor is to receive the images coming from the 
cameras, to process the images in order to determine whether or not persons or objects 
are present in the detection zone and as a function of the presence or absence of 
persons or objects in the detection zone to generate command signals for controlling 
the staircase, which signals are to be applied to the control device after a stop only 
when no obstacle and/or person is detected on the escalator and/or moving walk). 

Claim 3 is rejected under 35 U.S.C 103(a) as being unpatentable over Ponsot et 
al., US6, 606,538 in view of Konolige et al., US- 2005/01 00207A1 and further in view of 
as applied to claim 1 above and further view of Ahl et al., US-5, 704,464. 
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Re claim 3, the combination of Ponsot and Konolige fails to teach the monitoring 
system characterized in that the video cameras are located in a balustrade of the 
escalator and/or moving walk. Ahls discloses a passenger sensor (fig. 2:32) for a 
convey or includes a transmitter assembly that is positioned within a channel in the 
conveyor or balustrade (fig 2:22). Therefore, the combined teachings of Ponsot, 
Konolige and Ahls et al, as a whole would have rendered obvious to one of ordinary skill 
in the art at the time the invention was made to modify the arrangement of using a 
monitoring system located in the balustrade as claimed for the benefit to determine the 
presence of a passenger. 

Conclusion 

1 3. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to JESSICA ROBERTS whose telephone number is 
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(571)270-1821 . The examiner can normally be reached on 7:30-5:00 EST Monday- 
Friday, Alt Friday off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Marsha D. Banks-Harold can be reached on (571) 272-7905. The fax 
phone number for the organization where this application or proceeding is assigned is 
571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Marsha D. Banks-Harold/ 

Supervisory Patent Examiner, Art Unit 2621 

/Jessica Roberts/ 

Examiner, Art Unit 2621 



